Dealer: Dynamic Request Splitting for Performance-Sensitive Applications in Multi-Cloud Environments

Enterprises are increasingly deploying their applications in the cloud given the cost-saving advantages, and the potential to geo-distribute applications to ensure resilience and better service experience. However, a key unknown is whether it it is feasible to meet the stringent response time requirements of enterprise applications using the cloud. We make several contributions. First, we show through empirical measurement studies that (i) there is significant short-term variability in application workload and response times of individual components; however (ii) the response times of the same component in different data-centers are often uncorrelated. This leads us to argue that there are potential latency savings if work related to a poorly performing component is dynamically reassigned to a replica in a remote data-center. We leverage this insight to build a system that we term Dealer which for each component, dynamically splits transactions among its replicas in different data-centers. In doing so, Dealer seeks to minimize user response times, and takes component performance, as well as intra-datacenter and inter-data-center communication latencies into account. We have implemented Dealer in a way that it can be added to any multi-tier application. Evaluations of our approach on two multi-tier applications on actual Azure cloud deployments indicates the importance and feasibility of our mechanisms. For instance, the 90%ile of application response times could be reduced by as much as 6 times under natural cloud dynamics.